Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[8.16](backport #6199) Integration tests autodiscovery #6399

Merged
merged 1 commit into from
Dec 19, 2024
Merged

[8.16](backport #6199) Integration tests autodiscovery #6399

merged 1 commit into from
Dec 19, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Dec 19, 2024

What does this PR do?

Implement integration test autodiscovery using a plain go test call like go test -tags integration github.com/elastic/elastic-agent/testing/integration -v -args -integration.dry-run=true -integration.autodiscover.

The YAML output is written in a file (by default discovered_tests.yaml) and is structured like the sample below (some parts have been omitted to keep the sample short)

- os_family: windows    # This indicates on which OS type we need to run the test
  arch: amd64                 # This indicates on which CPU architecture we need to run the test
  os: ""                              # This indicates on which OS (by name) we need to run the test ("" means any OS for the given type)
  version: ""                     # This indicates on which OS version we need to run the test ("" means any version for the given type)
  groups:                          # List of test groups to run on the runner
    - name: upgrade        # Test group name
      tests:                           # List of tests belonging to the group
        - name: TestStandaloneUpgradeRollback # Test name
          metadata:
            local: false            # True if the test can be run locally
            sudo: true             # True if the test requires sudo
        - name: TestStandaloneUpgradeRollbackOnRestarts ...

- os_family: linux
  arch: amd64
  os: ubuntu
  version: ""
  groups:
    - name: deb
      tests:
        - name: TestDebLogIngestFleetManaged
          metadata:
            local: false
            sudo: true
        - name: TestDebFleetUpgrade
          metadata:
            local: false
            sudo: true
- os_family: linux
  arch: amd64
  os: rhel
  version: ""
  groups:
    - name: rpm
      tests:
        - name: TestRpmLogIngestFleetManaged
          metadata:
            local: false
            sudo: true
        - name: TestRpmFleetUpgrade
          metadata:
            local: false
            sudo: true

In order to filter the output we can use a tool like yq

➜  elastic-agent git:(integration-tests-autodiscovery) ✗ yq '[.[].groups.[].name] | unique' testing/integration/discovered_tests.yaml

image

Why is it important?

This is necessary to dynamically generate CI steps or provisioning for running the tests on the appropriate platforms

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

In order to generate the YAML file we can run

go test -tags integration github.com/elastic/elastic-agent/testing/integration -v -args -integration.dry-run=true -integration.autodiscover

this will generate a file in <repo root>/testing/integration/discovered_tests.yaml

It can be combined with the existing group and sudo filters (see -integration.groups and -integration.sudo)

Related issues

Questions to ask yourself

  • How are we going to support this in production?
  • How are we going to measure its adoption?
  • How are we going to debug this?
  • What are the metrics I should take care of?
  • ...
This is an automatic backport of pull request #6199 done by [Mergify](https://mergify.com).

@mergify mergify bot requested a review from a team as a code owner December 19, 2024 09:32
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Dec 19, 2024
@mergify mergify bot requested review from michalpristas and michel-laterman and removed request for a team December 19, 2024 09:32
@mergify mergify bot assigned pchila Dec 19, 2024
@mergify Mergify
Copy link
Contributor Author

mergify bot commented Dec 19, 2024

Cherry-pick of a7597b0 has failed:

On branch mergify/bp/8.16/pr-6199
Your branch is up to date with 'origin/8.16'.

You are currently cherry-picking commit a7597b01d.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   pkg/testing/define/define_autodiscovery.go
	new file:   pkg/testing/define/define_autodiscovery_test.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   pkg/testing/define/define.go
	deleted by us:   pkg/testing/define/define_flags.go
	deleted by us:   testing/integration/main_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@pazone pazone requested a review from pchila December 19, 2024 09:35
@pazone pazone removed the conflicts There is a conflict in the backported pull request label Dec 19, 2024
@pchila
Copy link
Member

pchila commented Dec 19, 2024

@Mergifyio rebase

@mergify Mergify
Copy link
Contributor Author

mergify bot commented Dec 19, 2024

rebase

☑️ Nothing to do

  • -conflict [📌 rebase requirement]
  • -closed [📌 rebase requirement]
  • queue-position = -1 [📌 rebase requirement]
  • any of:
    • #commits > 1 [📌 rebase requirement]
    • #commits-behind > 0 [📌 rebase requirement]
    • -linear-history [📌 rebase requirement]

@mergify Mergify
Copy link
Contributor Author

mergify bot commented Dec 19, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.16/pr-6199 upstream/mergify/bp/8.16/pr-6199
git merge upstream/8.16
git push upstream mergify/bp/8.16/pr-6199

@pchila
Integration tests autodiscovery (#6199)
6b95cea 
* Define output model and yaml output

* Add integration test autodiscovery flags

* Change assertions to ElementsMatch in define autodiscovery tests
@pchila pchila force-pushed the mergify/bp/8.16/pr-6199 branch from d9e2699 to 6b95cea Compare December 19, 2024 17:17
@pchila pchila enabled auto-merge (squash) December 19, 2024 17:24
@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

@pchila pchila merged commit e1e9971 into 8.16 Dec 19, 2024
14 checks passed
@pchila pchila deleted the mergify/bp/8.16/pr-6199 branch December 19, 2024 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pchila pchila pchila approved these changes

Assignees

@pchila pchila

Labels
backport
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pchila @pazone